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Remarks 

Claims 1-31 are pending in the application. Claims 1-31 are rejected. All 
rejections are respectfully traversed. 

Claim 23 is rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite. 

Claim 22 is amended to define the coordinates o(u, v) for the pixels in the 
output image used in claim 23, see paragraph [095]. Claim 1, line 10, 
provides antecedent basis for "viewing parameters" in claim 23. 

8, Claims 1 -4, 14, 16, 20-24, and 27-31 am rejected under 36 U.S.C. 103(a) as 
being unpatented* over Anderson (US 5, 714, 097) in view of Wenger et at (US 

As stated in the present application at paragraph [058], the system provides 
stereoscopic color images for multiple viewpoints without special viewing 
glasses. The claimed invention includes a three-dimensional display unit 
configured to concurrently display a plurality of output videos onto a single 
display surface according to viewing parameters. In contrast, the Anderson 
system uses a pair of stereo goggles having separate right-eye and left-eye 
displays. Anderson cannot use a single display surface. 

As per claim 2, the system uses a plurality of cameras to acquire calibration 
images displayed on the display surface of the three-dimensional display 
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unit to determine the viewing parameters. The cameras 112 described by 
Anderson with respect to Figure 7 acquire videos of the scene or coverage 
area 40, and not the display surface as claimed. Anderson describes 
calibrating those cameras. The cameras as claimed acquire images of 
calibrations images that are used to determine viewing parameters. 



As per claim 3, Anderson uses a video output device in the form of a pair of 
stereo goggles having separate right-eye and left-eye displays, see Figure 3 1 
and the description thereof. The system of Wenger projects non-overlapping 
side-by-side images of sub-scenes, see Figures 1-4, 6-9 and 11. Such a 
system would not work with the Anderson goggles. The Anderson 
stereoscopic goggles require that the left and right eye see approximately the 
same view to obtain the virtual reality effect, see the description of Figure 32 
at column 39: 

PIG. 32 OSttstftato. a* m esaaapl*. certain parameters mz4 
by Ite ri$t *ad MX rendearess, 23*2 aorf 2303. la isMsiog *5 
fetiriwgectfve JrigN:«eye and left-eye Moaes, For each eye, 
fee fjcwswlite is respectively <tft'm from the cumm virtual 
yscWjpcsBt t»y half the Mefocekr distance (i.e., &&tmx 
betwwa Hw «kswsf** eye*), m4 Reefed toward m ffcjetl 
bciof focused upon For exaiapJe. the right eye viewpoint S» 
24+2 is <sfff?t from ths virtual vkwjKjjst 14GI a tteuaxc* 

snd directed towards a pant of awagcat* 
Jr armnHt *n 88 t*j . i n gfooi > n net e 
viewport 24*3 is from the virtual vieupomt by s 
dbUKej^rMt^ee^ towawls the point of ctMtvergeftce *S 
24f? dssmnased by the object bcjsg focused upon. The 
oetXttpKdfeg vlemeg dkectwas f<r she right and left eye 
torn* «e ffc&obly cakuteted fey *e right «ad left re»- 

auffaee of eldest det*h which is aot **t of the bsekanxmd «> 



If the Wegner images were combined with the Anderson goggles, each eye 
would see a different sub-scene, and at most two sub-scenes can be used 
without the 3D effect. With all due respect, the Examiner's statement, 
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The advantage is that it adjusts t he picture quality for 
each of a plurality of images that comprise a picture and that distributes or balances the 
transmission of the plurality of images in order to optimize the overall picture quality in a 
video transmission. 

is a non sequitur. Projecting different images to different eyes using the 
Wegner projectors and the Anderson goggles has nothing to do with 
distributing and balancing the quality of video transmission. The 
combination of Wegner and Anderson would be a virtual mess and not a 
virtual reality. 

Organic LEDs as claimed are incompatible with the stereo goggles of 
Anderson. 

With respect to claim 14, as stated above the systems of Anderson and 
Wegner are incompatible with each other. 

With respect to claim 21, each video camera has viewing parameters that 
include a position, orientation, field-of-view, and focal plane. The Examiner 
states that Anderson does not teach parameters, but that it is "well known in 
the art" to use these viewing parameters. The Examiner provides no support 
for this assertion. MPEP 2144.03 states: 

"It would not be appropriate for the examiner to take official notice of 
facts without citing a prior art reference where the facts asserted to be 
well known are not capable of instant and unquestionable 
demonstration as being well-known. For example, assertions of 
technical facts in the areas of esoteric technology or specific 
knowledge of the prior art must always be supported by citation to 
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some reference work recognized as standard in the pertinent art. ...;In 
re Eynde, 480 F.2d 1364, 1370, 178 USPQ 470, 474 (CCPA 1973) 
('[W]e reject the notion that judicial or administrative notice may be 
taken of the state of the art. The facts constituting the state of the art 
are normally subject to the possibility of rational disagreement among 
reasonable men and are not amenable to the taking of such notice.')." 

3D television is still considered an esoteric technology. 3D television is still 
very much in the experimental stage, and commercial 3D television systems 
are not yet available. The field of 3D television is still being explored and 
what type of viewing parameters are required to provide plausible 3D 
images to viewers is very much an open question. Any number of factors 
could be considered. Certainly, the viewing parameters as claimed are not 
capable of such a demonstration. Certainly, that the viewing parameters 
should include position, orientation, field-of-view, and focal plane of each 
camera to accurately provide a 3D viewing experience is not instantly and 
unquestionably demonstrable. 3D television acquisition and rendering is a 
technically difficult area and assertions that certain procedures in the field 
are well known must be supported by evidence as required by the MPEP. 

Furthermore, MPEP 2144.33 goes on to state: 

"If such notice is taken, the basis for such reasoning must be set forth 
explicitly. The examiner must provide specific factual findings 
predicated on sound technical and scientific reasoning to support his 
or her conclusion of common knowledge. See Soli, 317 F.2d at 946, 
37 USPQ at 801; Chevenard, 139 F.2d at 713, 60 USPQ at 241. The 
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applicant should be presented with the explicit basis on which the 
examiner regards the matter as subject to official notice and be 
allowed to challenge the assertion in the next reply after the Office 
action in which the common knowledge statement was made." 

The Examiner sets forth no reasoning as to why he believes that viewing 
parameters as claimed are well known. The Examiner's statements: 

Howeves i s kn; m in ^ art tc j^l ,-» wing pa'-iirHt rs thai n * ! ' d« a 
position, orientation, field -of-view, and focal plana for a video camera (Official Notice), 
and 

Therefore, it would have beers obvious to me having ordinary ski in the art at 
the time of invention was made to modify the virtual reality television of Anderson with 
the method of Wenger to apply viewing parameters such as position, orientation,, field of 
view and focal plain of video cameras, 
are merely conclusions with no reasoning as to why the Examiner feels that 
these are well known. As such, the Examiner is depriving the Applicants of 
the ability to challenge his assertion, which is clearly in violation of the 
directives of the MPEP. 

The Examiner is respectfully requested to either provide evidentiary support 
and to clearly state his rationale in support of his assertions or to withdraw 
his rejections. 

As per claim 22, the controller determines, for each output pixel o(u, v) in 
the output video, a view number v and a position of each source pixel s(v, x, 
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y) in the decompressed videos that contributes to the output pixel in the 
output video. 

The Examiner refers to paragraphs [040], [041] and [043] of Wenger. 

Paragraph [040] describes sending data associated with images to the 
encoders. The method for the transmission of the video images can be by the 
means of either an analog or digital connection. A Bit Rate Control Manager 
receives a QP value for each encoder, and generates a new bit rate for each 
encoder. The bit streams (with bit rates as determined by the BCM) are then 
multiplexed with a multiplexer and distributed over a wide area network to a 
remote station. 

Paragraph [0041] describes a remote station 36. Four bit streams are 
received by a plurality of decoders. The typical video conferencing 
environment also permits images to be captured at the remote station for 
transmission back to the first station. 

Paragraph [0043] describes how each of the cameras is coupled to each 
digital signal processor and synchronized. All digital signal processors 
communicate to each other the target bit rate that was used when coding the 
last picture captured by the cameras. The DSPs also communicate to each 
other the achieved quality in the form of the average QP value for that last 
picture captured by the cameras. Each digital signal processor runs an 
identical algorithm in order to calculate the next target bit rate. This target or 
new bit rate is used in the encoding step of the next picture captured by its 
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respective camera, as the rate control parameter. Using the newly 
determined bit rate, the system and method cause the average QP value to be 
substantially the same, thereby insuring a substantially identical quality of 
the pictures that make up the broadband image. This process continues 
during the lifetime of a video conference so that the quality of the broadband 
image displayed at the station is of the best possible quality that the 
bandwidth of the network and various components of the system will allow. 
Note that the average QP is calculated by adding all QP values of all 
macroblocks belonging to the coded picture, and dividing it by the number 
of macroblocks in the coded picture. Note also, that the calculation of the 
new target bit rate can also be performed for a plurality of macroblocks 
while coding a picture, an entity called a slice in many video compression 
standards. Doing so, the accuracy of the method and system can be higher, 
but the computational demands are lower. It is also possible to increase the 
time interval between the calculations of the new target bit rate to a 
multitude of the inverse of the frame rate. In this case, the computational 
demands are lower. The quality measurement such as in the form of the 
average QP, is always performed using all those macroblocks that were 
coded in the time interval between the last calculation and the current 
calculation. 



None of the above three paragraphs indicate how source pixels are mapped 
to output pixels. The rejection of claim 22 is improper. 

As per claim 23, the Examiner merely relies on the improper rejection of 
claim 22, and does not consider the additional limitations of claim 23, which 
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considers a weighted linear combination of k source pixels according to 

k 

o (tt , v) = £ w.s(v,x,y) , where blending weights w, are 

;=o 

predetermined by the controller based on the viewing parameters. 
Paragraphs [0040-0043] do not describe a weighted linear combination of 
pixels. The rejection of claim 23 is improper since the Examiner fails to 
provide any prior art that teaches the claimed limitations. 



At paragraph [0038] Wenger states: 

In one embodiment of this invention described later 
herein, the average quantization factor or QF value of all 
m icrob ocks is also used hem bticaJl} as a i indication of the 
qualify of one of the coded pictures that art; optimized by the 
MCR< ' algorithm. Nor if at tht average OF is calculated by 
adding all QP values of ail macrocodes belonging to the 
c sdtd picti re-, a td di\ idi.og it by the number of macroblocks 
m the coded picture. 

There is nothing there about a block of source pixels contributing to each 
output pixel. 



As per claim 27, claimed is an arrangement of the cameras and an 
arrangement of the display units, which are substantially identical, and the 
number of cameras and display units is greater than two. Anderson is 
restricted to a single display unit with left and right eye display generator, 
see Figure 31. Such a system is incapable of operating with more than two 
display units. 



Wenger cannot be combined with Anderson. Anderson uses the principles of 
stereo television systems that use left/right camera pairs to generate separate 
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images to be displayed to the left and right eyes of a viewer to provide an 
illusion of depth; viewing position is still restricted to actual camera 
locations. Wenger uses a compositing of non-overlapping images, see Figure 
lb and paragraph [0013]: "The projector beam directions of all projectors PI 
to P4 are arranged in such a way that the four displayed sub-images II to 14 
spatially compose a full image I that geometrically resembles the captured 
scene A." 

As per claim 29, Wenger acquires a wide-band scene. As best as can be 
understood, in Wenger a video of a "wide-band scene A, with an aspect ratio 
of 16:3, consists of four spatially adjacent sub-scenes Al, A2, A3, and A4." 
That is, the "wide-band" applies to the spatial arrangement where the width 
is much greater than the height. The invention acquires high dynamic light- 
fields. A dynamic light field varies rapidly in the temporal domain. The 
Wenger spatial arrangement has nothing to do with the claimed temporal 
variation. 

Similarly, Wenger deals with the spatial arrangement of the projected 

images, see paragraph [0013]: 

I The projector beam, directions of all projectors FI 

to P4 are arranged in mob & way that the four displayed 
sub-images 0. to 14 spatially compose a full image I that 
geometrically resembles the captured scene A 

The dynamics of the claimed rendering has to do with temporal variation. 
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9. Claim® 5-13 are rejected under 35 U.S.C. 103(a) as being unpafemta&Ni w 
Anderson (US 5, 714, 997) in view of Wenger etal (U S 20Q4&1 79591), and in further 
view of Tung (US 2002/0122140). 



As per claims 8-9, Tung states: 

Hmv) \v<k Jiv jitkuu jiim iik'i x 

described using linearly pohi azi J oats rial v» each portion 

rj lilt -- Stetll tin u$ 0 l tl J I i I ] 1 l ed L ei'HS n 
L^lsd'sV UVtl ] 1 M vh ,A sW< «" I L I sj ( L i hit i p <] 32 

in .. pa re l bccli en * u3d be replace* y a f ssd « 
circnl ji 1 1 i ei 1 Iil ! 1 ik is fit i) the 

pre lerretl eroixKlimeni would have, hi jKkKs.km Us iht active 
ixmdor 20 sod th*. lirwar \*>htim 22 (mnx<i$\ tin; eye), § 
quarter wave etaeal is (he optical pad) at she location 
whsrt e :•><■:• om po an < existed it) Kit;. 1. 

Tung does not describe a passive display device that uses a flexible fabric as 
claimed. 



As per claim 10, the viewing direction of the viewer is not described by 
Tung: 

f(K»?ft 1 use of ltqi crystal shutter gl es ibi 5D 
s < ■ u[» icwiog is i tslt i I in l-UL 568 \s< ucn«* 
i ro&g&s sh:H ill* xuweisei lot i I i (hi \ t * 
i lisplaved 1 t vkwmg sen u f'he scr««n is 

I ..s [i I How i vtll x massed 1 clovv iht 
display ilevice srsay also lx- either a direct projection or rem: 
r |tvt hi t <_ i> ^ii'im: hi 64 »j> , 1-* tJk 

j.iiv>H\ ^iK 1 L iNjO 'ill 1 <fk VHI it, jd is i -> rt 

opcireei and closed so ibat Ok left eye staler is opesr wfaess 

1 t v - fiO W S[ 1 III llll I 1 <L ll Lit I 

Mi[r » v u \ „ if ^ 6S i < *i I U t 

si if - , ju ,d kL dh ,d 1 till U d r i [ ssiiij 

!l -i)Uh Hit d 5 Itth 1 ik ,'fll ^ ll II I >dl< 'M. \V II the 

ml rU _ . m i J J } Ur -Ml L r 

■.hs fi;tnsitt!>!5 s s rn shs cfi eyt vkw, to hi > ghi cyt \ ev, 
i'kjirn kf! to nght with llw led ey< cyele on tbt left of the 
[? tu 1 1 1 1 i 1 1 yen els n ih. >h . im In Hi 

iigure, tiai« increase from lefi to sighL 
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As per claims 10-12, paragraph [0036] above does not describe a display 
surface of a display unit disappearing when images are displayed on the 
surface. 

As per claim 13, Figure 2 of Tung and paragraph [0036], see above, of Tung 
do not describe acquiring images. The Figure only shows a CRT and lenses. 




10, Claims 15 and: 17-19 are rejected under 35 U.S.C, 103(a) as being unpatentable 
over Anderson {US 5, 714, 997) in view of Wenger eta! (US 2004/0179591), and in 
further view of Haskell et ai (US 6,055,012). 

As per claim 15, the claimed system has a plurality of video cameras that are 
in a regularly spaced linear and horizontal array. Haskell shows a bank of 
cameras each capturing a slightly different view of the scene. The cameras 
are assumed to be located side-by-side, however, in all generality, the 
cameras may be located in any configuration and each camera may have any 
orientation. Figure 2 shows cameras in a side-by-side configuration. The 
cameras are shown as a one-dimensional array or they can be in a two- 



18 



MERL-1538 
Pfister et al. 
10/783,542 



dimensional array. Haskell does not describe cameras that are regularly 



As per claim 17, in the claimed system, an optical axis of each video camera 
is perpendicular to a common plane, and the up vectors of the plurality of 
video cameras are vertically aligned. Haskell does not describe cameras with 
perpendicular optical axes. Instead, Haskell describes parallel optical axes: 

Moreover, 

there is no implied con-strain! to a parallel optical axis on the 
camera orientation geometry. While tlie arrangement of 
views can he application dependent, and thus very flexible, 
efficient coding can only be performed if mere is significant 
ei mtc latroo r etwee o ihe views. 



As per claim 18, the viewing parameters include intrinsic and extrinsic 
parameters of the video cameras. There is no description of extrinsic 
parameters at column 6. 

DUAL IP DESCRIPTION 

w 

EI* > I - , multi-vicv co i n di 
d^ndum mJ disomy vo . m ^emdj^i: in ursuisi m b< 
[| ns i ti da il i i wi tl in nf.i^ .J esmsN 
i'nm which ykkls dfiaem c-'mpjx^s.nai os (rumi-vistf w 
^ u easuring < mpk> i ot^t^iacs.! nco ng/dax>di.ns 
sad eoittpatibii ky with mxm vkko displays % scene UN) 

■^luvlp'JKd \ f k v.< t M th f I lU^ v M»d 

HI?, wlui .".tnTE.i HI! MW, 11^ niJ KM ..u urn .urn- 
hi hU c 1 1 iov- < the --cux md nulpJt '\[tl < 

2 ,, liiBs U2, 113 -mi 114 S in juiji 1 ni s u v i v. i--t 
\u * i-- -l kd ts ( v ini j i- s i-h i i\ ir ti^ t 0 Ji 1 
difterest vk% ol ibt scene dih ugh u gene u, t rat 
uu\ h i< < NJthj i 1> sin t k I k f ujti in i f <m 
1 k s YK 1 Ihi * irm r ts ik t-- un< > ■ K l d sid s 

,. ; stds h >vmu n- i . js.^ ldm ttk unttis l» 1 v.u 

Hi i V J ll II ill K li. xiliKfi H fliV \U\ in - 

Similarly, there is no description of extrinsic parameters at column 7. As 
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known in the art, extrinsic parameters of a camera describe the position and 
orientation of the camera using a rotation matrix R and a translation matrix 
t. There are three parameters for the rotation, another three for the 
translation, see paragraph [078]. The claimed viewing parameters extend 
extrinsic parameters to 3D television system. Haskell does not teach, show 
or suggest extrinsic parameters for 3D television, at columns 6 and 7, or 
anywhere else. 



As per claim 19, the claimed system includes means for selecting a subset of 
the plurality of cameras for acquiring a subset of videos. The Examiner cites 
Wegner col 7, line 40, col 8 lin 5: 

[«M*4«1 A plurality of cameras el, C2, C3 aad C4 capture 
mi-.. 2«. 22. 24 «s«1 26 am* scsate the stela 

< led v, US! i iges 2<>-2ft lo iSif tiK.-n tl til-i n4> 

12-1S, ucftptclivelv 'i'he mi'lbod 1W lb© transmission of the 

il i to i 1 [ i- i I 2 t !h j j ^ t> P i. 

l onun] Massagcr pintew C'BrMK sct.emsn,;,^ irhsstoiUed 
.so ol,., k 3I> hi I fti. 2, winch rctLiv^.s .i OP vsshie tor ta* 
e aaiifcr enl-i£«4 and acmat «>it Me lor each on^c capuued 
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o-. <i bv dc.-.ni!.Uisr*k-«t:.r3?t which slesoishipiexeo itn. ejun.,h 
too I'-.su. bit. streeoso in. the illosstmtkn.. This, o.suf bit. sSs-ee.ms 
s so. iTct-ixott hv a pjuralilv fit' det-mfcfs. : ' I •. i 4u ! H ' 242. 
PBC344 ;t»d i >.i < 44 fi soseh i g a i.ssutci ed doii't.d 
-i, > 1 oo v. -v-.u 4*1 t , 42,. 44- md .. n-,! t i'vl\ Int 
v. of ithrs-foition usui s.i iifif; fsf a fidi H it: invention te 
civ-scribed usiw* tncofte }.\-H.1.-f.';..i4l.2-.}.8 sod deaxfcrs 
['>IH'" L-j;>L;C.440-4<j-, but st shs-Msid be undo retook that a fewer 
i is- •.'.ts .-.K.s oti-ol .<-■• c.o: in: sts,td . provided 5 here is >ti. so., than 
juo; one, ASsie the uiveoikin csss be tis» »l is .> corns-msee; 
im'e5j.r.-!;csd erht eno >tier/de-c(.idcr cownKMsly kwwn os 
"oxfecf,." .i-'eii- exfioiple, is shosdd he oofkootooes tfusS while- 
Ui ^.tn Tv. I i i ' iji n! v i i l mi ^ i I nit i ■- 

£rom the first .station 11 iihrsirated hi Fit;. 2. the iypicaS 
vkSeo es.nde.rerseks.si wviiwimeia win also (mmti imagea So 

the t -til, u 11 . 

[*Sf*42] It should be iiusiei'steso-d dsas. (be hit raits oxre- 
spsotote to the bit nste eorrespoodisio to the iniii>es captured 
-, vt us 3 i M> ^ < s. t , ^ tli the Ol .ah k,. 
esdi , t the >. i shire v .its i * 4« 5i) ^2 h.J ^4 i- id n ion 
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rt*0431 Keforrmss to FIG. 2. it shouid hi: ui-derstood that. 
•: 1 1 of the c-ssvseras. C't , « 7:2, f.3 ami < 4 ;i f e twwpfcd To each 
digstJiS fei^iUiJ p*oco-«soir (DSP.) 12a„ 14**, Kitf., and It 
showkS be J:onhcr i.!ixfcrsuKii.l, however, i:h<sf l:bc e»*»er»s 
CJ C4 at- v,>!Ll>mi>\ tJ J! ti v by r jrsoVKh;sg ;s 

boo-jscmtal isrsc) vevSk.-55.S syyK;brom;\;>i.AoY.i *jiin;t5 ssroj ptxel 

There is nothing about selecting cameras in the above paragraphs. 

1 1 . Claims 25 and 26 rejected under 35 U,S.C. 103(a) as being unpatentable over 
Anderson (US 5, 714, 997) In view of Wenger et al (US 2004/0179591), and in further 
view Of Morisntta (US 4, 737, S4G). 

Claims 25 and 26 specify a three-dimensional display unit that includes a 
display-side lenticular sheet, a viewer-side lenticular sheet, and a diffuser. A 
substrate can be between each lenticular sheets and the diffuser or between 
the lenticular sheets and the reflector. 

First as stated above, it should be noted that Anderson uses a video output 
device in the form of a pair of stereo goggles having separate right-eye and 
left-eye displays, see Figure 3 1 and the description thereof. The Examiner 
states: 

Therefore, it woutd have been obvious to on© having ordinary sfc'tii in the art at 
tie time of invention was made to modify modified the virtual reality television of 
Anderson according to Morisbita's invention to incorporate features recited in claim 28 
because it provides brighter pictures and widens the range of viewing angles, 

It should be obvious to one of ordinary skill in the art that the system as 
described by Anderson can never work with a lenticular lens as described by 
Morishita. The combination of Morishita and Anderson results in an 
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inoperable system. The separate view goggles of Anderson cannot 
incorporate the lenticular lens of Morishita. 

Furthermore, even if Morishita could be combined with Anderson, Morishita 
only describes a single lenticular lens. Claimed are two lenticular sheets, one 
is a display-side lenticular sheet, and the other a viewer-side lenticular sheet. 
There is nothing like that in Morishita. 

As shown in Figures 10-13, the substrate 141 is neither between the 
lenticular sheets and the diffuser nor between the lenticular sheets and the 
reflector. The substrate 141 in Morishita in all embodiments is on an outside 
surface. 

m to FIG. 12 



In addition, Morishita does not show or describe a reflector. 

It is believed that this application is now in condition for allowance. A 
notice to this effect is respectfully requested. Should further questions arise 
concerning this application, the Examiner is invited to call Applicants' 
attorney at the number listed below. Please charge any shortage in fees due 
in connection with the filing of this paper to Deposit Account 50-0749 . 
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Respectfully submitted, 

Mitsubishi Electric Research Laboratories, Inc. 

By 

/Dirk Brinkman/ 



201 Broadway, 8 th Floor 
Cambridge, MA 02139 
Telephone: (617) 621-7517 
Customer No. 022199 



Dirk Brinkman 
Attorney for the Assignee 
Reg. No. 35,460 
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